This invention relates generally to the field of mass storage and specifically to the field of managing memory resources, especially on write-once media, for such storage.
Various techniques have been used to manage memory and memory resources. Some of these techniques are described U.S. patent application No. 136,979, filed Dec. 3, 1987 and now abandoned, and U.S. Pat. application No. 053,823, filed May 26, 1987, now U.S. Pat. No. 4,864,572 issued Sep. 5, 1989, both of which are assigned to the Assignee of this application and of which are incorporated herein by reference. As those applications explain, several problems arise when trying to store very large amounts of data, and many more problems arise when using write-once media, such as optical discs, for storage. One problem is organization, another is accessibility, and a third is limitations of storage space.
Before the advent of optical discs, large amounts of data were often stored on magnetic discs. Early magnetic disc systems were subdivided into small storage compartments or partitions. The size of these storage partitions was determined by physical constraints, such as the sizes of surfaces, tracks, or cylinders, and the partitions all resided within a single disc drive.
Some systems later allowed the operating system to manage the placement of these storage partitions first within a single disc drive and then among several drives. Even in these later systems, however, operators needed to specify partition boundaries. Storage systems eventually evolved which were able to alter the size of these partitions dynamically. This advance led to the development of storage management utilities that rearranged the written storage within the physical media to balance the access load of different magnetic disc drives.
One advantage of systems which had storage partitions defined according to physical specifications was that the loss of a disc drive, whether from failure or regular maintenance, did not prevent the computer system from continuing to run, unless of course the storage on the disc drive contained critical system information. Another advantage was a significant ease in determining what storage was lost while the drive was disabled.
Unfortunately, these systems demanded considerable attention and consumed significant resources whenever space and performance needs changed or whenever the initially estimated requirements were incorrect. In such circumstances, storage allocation and maintenance personnel needed to step in and reallocate storage partitions in such systems. Early IBM systems were estimated to require one storage management person for every ten (10) Direct Access Storage Devices (e.g., discs).
Some of the problems with conventional systems were ameliorated by treating several volumes as a single volume by storage or by management techniques that constantly reconfigured the memory. Other problems have become less important as the reliability of storage devices improved, thereby reducing the need to disable drives for maintenance and repair. These improvements, however, are limited to fixed quantities of storage which can be erased and rewritten, such as magnetic tapes and discs.
Despite these improvements, conventional storage systems are still limited in several respects. For example, storage management systems are not extendable, except for certain tape libraries in very limited applications. Therefore, storage management personnel are required to be intimately involved with data storage decisions.
If the storage system uses write-once media, such as optical discs, the problems are compounded because the magnetic storage techniques which move or adjust storage after it is written cannot be used. There is also much greater overhead, both in performance and space, in updating files on write-once media than on magnetic media. For example, magnetic disc systems usually store only active data; less active data is removed to a different storage for back-up and archives. With write-once media, however, the removal of infrequently used data requires physically removing the volumes on which the data is stored.
In addition, the size of magnetic disc systems is generally fixed. To gain free storage, old files are simply erased. Because write-once files cannot be erased, additional volumes must be added to gain free storage.
Adding new volumes is not without problems. Eventually, the entire system, sometimes called a "volume library," will become full and some of the volumes must be removed. If the storage is not managed wisely, the storage system will be very inefficient and require frequent loading and unloading of volumes.
Write-once storage has additional design constraints not shared by magnetic disc systems. Access to files on write-once media is fairly slow, and thus is most efficiently used with infrequently accessed data. For these reasons, it is important to minimize the management overhead of write-once storage.
The difference in media also manifests itself in the allocation of files into which the data is written. In magnetic disc systems, the location of free storage is not critical because storage can be reorganized to optimize the placement of data even after the data is written. This is especially important when free storage is later at a premium. In the write-once systems, however, the allocation of file storage and placement of the data is critical because generally the data cannot later be moved. Therefore, efficient storage in systems of write-once media requires algorithms which search for and allocate free storage based on detailed knowledge of the media, of the characteristics of the files, of the relationships between files, and of future storage requirements.
The storage allocation difficulties are exacerbated if the stored data is to be modified. In magnetic disc systems, updates are carried out merely by overwriting the file. Most of the time such overwriting requires no additional free storage. In write-once systems, however, files cannot be overwritten, so modifications consume additional storage. Additional problems arise if a file needs to be modified but resides on a volume of write-once media that does not have enough free storage to hold the modifications to that file. In this case, storage must be formed somewhere else in the system, preferably in a location which promotes efficient access.
Another constraint with current optical drive technology is that the two surfaces on each volume cannot be accessed simultaneously. Accessing the surface not currently under the reader requires removal of the disc from the drive, turning it over, and reinsertion. In constrast, most magnetic drives allow multiple surfaces to be read simultaneously.
Finally, it is desirable that concurrent applications be able to use a single write-once memory system. This is a problem because the write-once characteristic makes it difficult to use pointers which magnetic disc systems use to manage multiple applications, and the lifetimes of write-once storage make the use of pointers unattractive.
Therefore, it is an object of this invention to provide a method of managing storage which requires low overhead and which reduces the need for intervention of storage system personnel.
It is another object of this invention to provide a method of efficiently managing files stored on a write-once memory system.
It is a further advantage of this invention to improve the efficiency of storage by taking advantage of characteristics of the data to be stored.